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the quality cf information and services provided; (2) dissemination 
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problem areas; and (4) programmatic changes made. Survey responses 
were received from 88 percent of the school administrators 
participating in the 1978 assessment and are presented. EQA was found 
to play an important part in making decisions in areas such as 
changes in courses and teaching strategies. Often principals used EQA 
informaticn tc make decisions leading to revisions of eacisting 
prcgiams or the development of new programs. Many respondents 
indicated that EQA provided data that confirmed a problem existed or 
called attention to problem areas not previously noted by the 
district staff- "Percentiles on student performance areas" and 
"scores compared to predicted range" were selected as the most useful 
type of scoring and statistics used in the school report, although 
all six options were selected by at least 50 of the 99 school 
administrators- {Author/EL> 
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ABSTRACT 

A follow-up survey was conducted of the 1978 Educational Quality 
Assessment (EQA) to ascertain what impact the Pennsylvania Statewide 
Assessment Program -had on schools and school districts. The survey 
instrument consisted of 20 items tapping: the qxiality of information 
and services provided, dissemination activities engaged in by the district, 
V£j.ue in identifying problem areas, and programmatic changes made. 
Survey responses were received from 88 percent of the school administrators 
participating in the 1978 assessment. EQA was found to play an Important 
part in making decisions in areas such as changes in courses and teaching 
strategies. Often principals used EQA information to make decisions 
leading to revisions of existing programs or the development of new 
programs. Many respondents indicated that EQA provided data that confirmed 
a problem existed or called attention to problem areas not previously 
noted by the district staff. 



INTRODUCTION 

During the 1977-78 school year, Pennsylvania's Educational Quality 
Assessment (EQA) initiated new Instruments for all goal areas. The 
expanded instruments incorporated a stronger emphasis on the cognitive 
areas and a slight decrease in the affective areas. Also, the number of 
total items was increased to provide broader assessment coverage within 
each goal area measured. Along with the Increase in items, matrix 
sampling was first introduced during the 1977-78 school year. Matrix 
sampling permitted eighth and eleventh grade students to complete the 

assessment questionnaires in one sitting of approximately two hours; 

whereas fifth grade students needed approximately two and one-half hours 

to complete the assessment—frequently administered over a two-day 

period. 

Of Pennsylvania's 505 school districts, 113 volunteered to participate 
in the 1977-78 asseslnent. The number of schools and students assessed 
within the districts was: 

Number Number 



Grade of 

Level Schools 

5 453 
8 151 
11 128 
Total 732 



of 

Students 

29,956 
30,876 
28.653 
89,485 



All participating districts received their school reports-, from 
Division of Educational Quality Assessment staff members during the fall 



of 1978. 




PURPOSE OF THE SURVEY 
EQA is designed to offer reliable statistical information about 
strengths and weaknesses of schools upon which school officials can base 
sound educational planning decisions. Schools are encouraged to use the 
data and results as they wish. EQA's major function is to provide 
schools with a point of reference for Initiating a self -analysis of 
their program strengths and/or weaknesses. 

EQA follow-up surveys are deemed Important in order to ascertain 
what impact the data and information provided by the Division of EQA 
have on local school programs. Survey results were valuable for EQA in 
planning to better meet the needs of individual schools and school 
districts. 

PROCEDURE 

This follow-up survey was conducted among districts that participated 
in the March 1978 assessment. In August 1979, the Division' of Educational 
Quality Assessment mailed a 20-item follow-up survey to superintendents 
of the 113 school districts that participated in 1978. Replies were 
received from 99 school districts, or 88 percent of the districts, by 
the end of November 1979. 

A copy of the Follow-Up Survey is found in Appendix A to provide 
the exact item wording. Questions in the survey focused on: (1) extent 
of dissemination of EQA results, (2) usefulness of EQA data, and (3) 
contribution of EQA's "Resources for Improvement" packets. 
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FINDINGS 

Through the 1978 follow-up survey it was reported that most districts 
received the initial interpretation of the EQA report at a school district 
site. The initial interpretation of the EQA report was received by 
districts as indicated by the responses to Quostion 1. 

QUESTION 1. Initial interpretation of the EQA report was 
received: 

°E£i2H£ Number 

At a regional workshop at (Shlppensburg, Clarion. 
Harrisburg). 25 

In your district by an EQA representative. 72 
At a regional workshop and at a district site. 2 

n = 99 school districts 

n: number of school districts that responded to this question. 

Following the first question, respondents were given instructions 
to respond to questions 2 and 3 if the school report was delivered at a 
regional workshop. If respondents indicated the school report was 
presented at a district site, they were then asked to answer questions 4 
and 5. 

Information on the knowledge gained through a regional workshop was 
gathered by question 2. All but one of the 27 respondents indicated the 
individuals attending a regional workshop were able to interpret the 
school report to the remaining staff. 

Eighty-nine percent of the respondents were positive in their 
response to Question 3 on selecting a workshop presentation m the 
future. Comments made on this item are found in Appendix B. In general. 
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the regional workshops allowed participants to gain sufficient knowledge 
to interpret reports to school district staff members and would be the 
option selected in the future. 

Two questions were presented to the 74 school districts that received 
their school report at a local site. Question 4 asked if the representa- 
tive from EQA was thorough in their explanation qf the EQA report. All 
respondents were positive on chis item, thus indicating the EQA staff 
members are providing sufficient information when delivering school 
reports. 

The other question for districts receiving the EQA report at a 
district site was on selecting this type of interpretation in the future. 
Except for one respondent, all responses were "yes." School districts 
reflected favorable attitudes in their comments on this item which are 
found in Appendix C. 

A question was presented on the results provided by the EQA Student 
Questionnaire. Of the responses received. 98 percent indicated the 
results were either excellent or acceptable. This revealed a high level 
of satisfaction with the results provided by the EQA questionnaire. 
Comments on this item are found in Appendix D. 

QUESTION 6. Did you find the results provided by the new 
EQA Student Questionnaire to be: 



QPtion Number 



Excellent 35 
Acceptable 52 
Unacceptable 1 



Percent 

36.4 
61.6 
1.0 



1 (omit) 1,0 



n = 99 
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The most useful type of scoring and statistics used in the school 
report were identified from a list supplied with Question 7. "Percentiles 
on student performance areas" and "scores compared to predicted range" 
were selected most frequently. However, all options were selected by at 
least 50 of the 99 respondents. 

QUESTION 7. Which of the following items in the report did 

you find useful? (Check as many as appropriate.) 

OP^i-o^ number Percent^ 



Percentiles on student perfonuance areas 87 87 9 

Scores compared to predicted ranges 89 89 Q 

Condition variable data 50 5 

Item data for condition variables 58 58*5 

Criterion-referenced Information 55 55*6 

Item data for student performance areas 56 66 7 



n «= 99 
a 



when totaled will exceed 100 percent due to school 
districts selecting more than one option. 

Information was gathered on how easy the school report was to 
understand and whether the report provided sufficient information. 
Respondents selected a point on a continuum that traversed from "easy to 
understand" to "difficult to understand." Most replies indicatec the 
school report was easy to understand. Respondents also selected a point 
on a continuum of "not sufficient" to "too extensive"— based on 
information presented in the school report. There was a slight tendency 
for replies to favor the "too extensive" end of the continuum. In 
general, the responses indicated school reports were easy for most 
people to understand and captained an appropriate amount of information. 
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QUESTION 8. The information presented in the school report 
is: 



Easy to Understand 



Difficult to Understand 



Option Part 1 



1 
2 
3 
A 
5 
6 
7 



Number 




19 


19.2 


44 


4: . i 


20 


20.2 


7 


7.1 


4 


4.0 


2 


2.0 


0 


0.0 


3 (omits) 


3.0 



99 



Not Sufficient 



Too Extensive 



Option Part 2 

1 
2 
3 
4 
5 
6 
7 



Number 


Percent 


1 


1.0 


1 


1.0 


4 


4.0 


56 


56.6 


17 


17.2 


4 


4.0 


0 


0.0 


16 (omits) 


16.2 



99 



Through the follow-up study, it was also found that EQA results 
were widely dlssemi..ated. The assessment results were disseminated to 
various categories of publics and organizations with the following noted 
most frequently: srhool board members, principals, central office 
staff, and teachers. Under "Other." several groups were listed Including: 
long-range planning committee, citizens' advisory committee, and the 
community through local news media. 
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QUESTION 9. To whom hav-i your assessment results been 

disseminated? (Check as many as appropriate.) 

Option 

School board members 
Principals 
Central office staff 
Most elementary teachers 
Most middle/junior H.S. teachers 
Most high school teachers 
Local service clubs (Lions, Jaycees, 
etc.) 

PTA, PTO, any parent group 
Students 
General public 
Other 
None 



numoer 


Percent* 


Oft 

yo 


yy.U 


oo 


100.0 








OA O 


83 


83.8 


87 


87.9 


4 


4.0 


47 


47.5 


17 


17.2 


67 


67.7 


11 


11.1 


0 


0.0 



n = 99 

^Percentages when totaled will exceed 100 percent due to school 
districts selecting more than one option. 

It was found that EQA information is being widely shared with 
groups through different dissemination methods. Dissemination strategies 
included many different approaches such as newsletters, press releases, 
in-service presentations, school board meetings and faculty meetings. 
The two methods used most frequently were school board meetings and 
faculty meetlms. Under the option of "regular meeting with" respondents 
included: long-range plan committees, administrators and citizen advisory 
committees. 
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QUESTION 10. What methods have been used to inform others 
about the EQA results? (Mark as many as arc 
appropriate 0 

Option 



Faculty memorandum 

School district newsletter 

Press release 

Special written report 

Curriculum bulletin 

In-service presentations 

S^ihool board meeting 

Faculty meeting 

PTA presentation 

Special presentation 

Regular meeting with 

Other 
None 



Number 


Percent* 


28 


28.3 


53 


53.5 


56 


56.6 


22 


22.2 


4 


4.0 


65 


65.6 


90 


90.9 


85 


85.8 


34 


34.3 


21 


21.2 


12 


12.1 


4 


4.0 


0 


0.0 



n - 99 

^IVttTt^^^ when totaled will exceed 100 percent due to school 
districts selecting more than one option. ' 

Over 88 percent of the districts Indicated that EQA information was 
used by building administrators to make decisions about their buildings 
and programs. Also, over 70 percent of the districts indicated that~as 
a result of EQA information-revisions of existing programs are underway. 
Only 3 percent of the school districts reported that they had not, as 
yet. used the EQA data. Information usage varied in the responses for 
"Other" from confirming results of other tests to stimulating an evaluation 
of problem areas. 
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QUESTION 11. Which of tb€ followtng dMcrlba the um made 
of the BQA Infonetlont CCheck mm mmsxf mm 
appropriate O 



Option MuBiber Percent^ 

The inforaation hae not, aa yet» been 

used. 3 
The inforaation was used to favorably 

reflect our present programs. 53 53.5 

The information vas used by Individual 

principals to make decisions about 

their buildings and programs. 88 88.9 

The information served as a basis for 

teacher in-service activity. 66 66.7 

Revisions of some existing programs 

are underway as a result of infor- 
mation. 70 70.7 
A new program is being planned for 

one or more of our schools as a 

result of the information. 24 24.2 

A new program has been "tried out" 
in one of our schools aa a result 

of the information. 13 13.1 

A new program has been incorporated 
into one school's program as a 

result of the information. 9 9,1 

A new program has been incorporated 
into several of our schools as a 

result of the information. 10 10.1 

Other: 



7 7.1 



n « 99 

^Percentages when totaled will exceed 100 percent due to school 
districts selecting more than one option. 

More than three out of five districts felt that BQA data had an 
Influence on "changes in course content" and "changes in r^^M ng 
strategies." Based on responses to Question 12, school districts are 
often influenced by EQA johezx making such changes. 
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QUESTION 12. Which of the following can you attribute, at 
least in part, to information received from 
£QA? (Check as many as appropriate.) 



Option 


Number 


Percent* 


Changes in course offerings 


34 


34,3 


Changes in course content 


63 


63.6 


Changes in caaehing strategies 


60 


60.6 


Changes in teaching assignments 


12 


12.1 


Changes in financial allocations 


17 


17.2 


Changes in school facilities 


9 


9.1 


Other changes: 


6 


6.1 



n = 99 

^Percentages when totaled will exceed 100 percent diie to school 
districts selecting more than one option. 

Sixty-eight districts responded to the item on their teachers 
making use of EQA data. Of the districts responding, the average district 
percentage for teachers making use of EQA data was about 52 percent. 
All school districts responding to the item reported at least some 
teachers making use of EQA information. Respondents not completing this 
item often noted they were unsure of the correct percentage of teachers. 

QUESTION 13A. What proportion of your teachers have made use 
of the EQA data? 

- Percent School Districts 

Reported Reporting In the Ranpe 

5-20 7 

25-40 16 

45-60 23 

65-80 15 

85-100 7 

31 Comits) 

n = 99 
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A second part of Question 13 asked if greater teacher utilization 
of data was desirable. Two-thirds of the school districts reported that 
greater teacher utilization of the EQA data would be desirable. This 
figure probably would have been even higher if the 16 districts omitting 
the item had responded. 

QUESTION 13B. Is greater teacher utilization of data desirable? 

Option Number Percent 

Yes 66 66.7 

No 17 17,2 

16 Cpmits) 16.1 

n 99 

The last part of Question 13 was concerned with how EQA can assist 
in having teachers utilize data. Comments from school representatives 
are recorded in Appendix E. There were many different attitudes on this 
item. Comments ranged from this being a local function to requesting 
in-service work by the Educational Quality Assessment staff. Many 
districts favored using EQA resource packets and EQA staff interpretation 
sessions. 

School district employees often suspect that problems exist, but 
they do not have reliable data to support those contentions. Over 82 
percent of the districts reported that EQA provided them with data that 
confirmed a problem existed, or that EQA data called attention to 
problem areas not previously noted by the district staff. This was 
Important to the EQA staff, since assisting school districts in confirming 
problems and locating problem areas are an essential part of the assessment 
. program. 
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QUESTION 14. The EQA information: 

Option Percent 

a. Called attentioa to a problem area not previously 
noted by district staff. 9,1 

b. Confirmed suspicions about district problems. 45.4 

c. Did not identify any serious problems. 17.2 
Both a and b* 15 2 
Both b and c. loll 
Both a and c. 2.0 
a, b and c. 2.0 



n = 99 

Of the 99 school districts responding, 98 considered the EQA program 
to be at least "useful" In helping to make decisions. In addition, over 
30 percent of these districts indicated EQA was 'Very useful" in helping 
rhem to make decisions. 

QUESTION 15. How do you consider the EQA program as a means 
of helping you make decisions? 

Percent 



Option 


Nusaber 


Very useful 


30 


Useful 


67 


Not very useful 


2 


Useless 


0 



30.3 
67.7 
2.0 
0.0 



E = 99 



Information was gathered on a series of publications made available 
to school districts which were called "Resource for Improvement" packets. 
Exactly one-half of the districts responding indicated they had requested 
at least one of the packets. This indicates that the single memorandum 
offering the packets, perhaps, was overlooked by many districts. Many 
school districts did, however, attempt to implement follow-up work based 
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on an analysis of their EQA results. Only districts that requested 
resource packets were told to answer Questions 17 and 18. Ml districts 
were instructed to respond to Questions 19 and 20. 

QUESTION 16. Did you or your staff request any of these 
packets? 

Option Number Percent 

Yes 48 48.5 

No 48 48.5 

—■ 3 Comits) 3.0 



n = 99 



Over 79 percent of the 48 districts requesting resource packets 
made use of the information. For respondents who indicated they did not 
make use of the packet, an additional question was posed— -"Why?" The 
responses are found in Appendix F for the first part of Question 17. 

QUESTION 17. Were the packets used? Yes No If no, 

why? 

Qpt^o" Number Percent 

Yes 38 79.2 

No 7 14.6 

3 (omits) 6.2 



n = 48 



Districts that responded with "no" in part 1 of Question 17 were 
asked in part 2 if they expected to use the packet ±n the current school 
year. The responses for part 2 are found in Appendix G. Only one 
school district indicated the resource packets would not be used in the 
current school year. All others indicated they would be used. 
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Districts that responded with "yes" in part 1 of Question 17 were 
asked in part 3 vho used the packets and how they were used. Administra- 
tors, curriculum coordinators, counselors, teachers and students were 
listed as having used the resource packets. How the packets were used 
varied, but most responses were related to curriculum work. Comments 
for part 3 are foiind in Appendix H. 

Question 18 asked which part of the packet respondents found 
useful. Part 5, "Classroom activities and Improvement strategies," was 
most often indicated as being useful, but the other parts were almost as 
frequently marked as being useful. 

QUESTION 18. Which part of the packet did you find useful? 
(Check as many as appropriate.) 

P^t Content Number Percent^ 



1. Goal statement, goal rationale, 

measurement rationale, scale 
description and scoring 15 

2. Validated projects 14 

3. Selected resources I7 

4. Practices and programs from EQA 

selections of long-range plans 17 81.0 

5. Classroom activities and improve- 

ment strategies 18 85.7 



76.2 
66.7 
81.0 



n = 21 

^Percentages when totaled will exceed 100 percent due to school 
districts selecting more than one option. 

QUESTION 19. What type of assistance would you like from PDE 
to supplement the initial EQA Interpretation 
session? 

f 

Twenty-two districts responded to Question 19. About 40 percent of 
these districts Indicated they were satisfied with the assistance 
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provided by EQA, or that no assistance uas required. The other comments 
dealt with various follow-up activities including on-site visits, providing 
specific suggestions and workshops for administrators. All comments are 
found in Appendix I. 

QUESTION 20. Comment on the value of EQA. 

Over 90 percent of the 82 districts responding to this question 
indicated EQA provided them with valuable information. Districts commented 
that EQA was important in formulating short and long range plans and 
that EQA assisted in evaluating both affective and cognitive areas. 
Most of these districts felt the EQA program was edacatlonally sound, 
useful and relevant to their programs for educational change and develop- 
ment. All responses for this item are found in Appendix J. 



CONCLUSIONS AND RECOMMENDATIONS 
Ninety-nine of the 113 school districts that participated in the 
assessment responded to the follow-up survey. The survey was designed 
to gather information on the EQA program and to obtain suggestions for 
program planning. 

School districts were satisfied with interpretation sessions at 
both regional workshops and local school district sites. Almost every 
respondent indicated the EQA staff member thoroughly explained the 
material at the interpretation session. Only three of the districts who 
participated said they would not select a regional workshop for future 
interpretation sessions. 



ERIC 



Id 



- 16 - 



The results provided by EQA were categorized as either "Excellent" 
or "Acceptable" by 98 percent of the school districts. This general 
acceptance of results was substantiated by districts indicating approval 
of statistics used in school reports. 

EQA information continues to be widely disseminated to many -persons , 
agencies, mass media and other organizations via many different methods 
and techniques. 

A majority of the districts indicated that the EQA program was 
helpful in conducting a needs assessment and providing them with pertinent 
information and comparative data which were not otherwise available. 
EQA was an important part of district decisions on changes in courses 
and teaching strategies. EQA was often used by principals to make 
decisions about their buildings and programs, resulting in revisions of 
existing programs or the development of new programs. 

EQA was considered to be "Useful" or "Very useful" in decision 
making by 98 percent of the districts. This was emphasized when over 66 
percent of the respondents indicated greater teacher utilization of the 
data was desirable. It was reported that about 52 percent of the teachers 
use EQA data. 

About one-half of the districts requested resource packets for 
selected goal areas. Most districts reported the packets were used and 
that the "Classroom activities and Improvement strategies" section was 
most useful; however, all other sections of the resource packet were 
considered to be almost as useful. 
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Several districts indicated they would like assistance from PDE to 
stipplement the initial interpretation session. Suggestions for assistance 
included the following: schedule visitations by PDE curriculum specialists 
provide specific suggestions and recommendations to improve weak areas, 
identify schools that have excellent programs and provide in-service 
programs. Other districts indicated the follow-up activities provided 
by EQA were satisfactory or that no additional assistance was required. 

Comments on the value of EQA indicated an over^elmlng positive 
attitude toward the program. EQA was considered to be helpful when 
evaluating school programs that are not evaluated by regular testing 
instruments. In general, the 1978 EQA program was successful in meeting 
its stated purposes and in providing valuable assessment services to 
participating school districts in Pennsylvania. 
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EDUCATIONAL QUALITY ASSESSMENT (EQA) 
FOLLOW-UP SURVEY 
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District 
Name 



Position 



Educational Quality Assessment (EQA) 
Fellof^up Survey 



1. Initial Interpretation of the EQA report vas received: 

a. at a regional workshop (Shlppensburg, Clarion or Harrisburg) 

°' to yottT district by an EQA representative. 

If a, answer questions 2 and 3. If b, go on to Items 4 and 5. . 

2. Did the persons attending the regional workshop return with sufficient 
knowledge to interpret the reports to the remaining staff? Yes 

N o If no, vhy7 



3. Would you choose this type of Initial Interpretation in the future' 
Yes N o Why? 



Go to, Item 6. 

4. Was the EQA representative thorough anough In his/her explanation of the 
report? Yes N o 

5. Would you choose this type of initial Interpretation in the 

future? Yes No Why? If ju,, what other methods would you 

suggest? ' 

6. Did you find the results provided by the new EQA Student Questionnaire 
to db: 

a. excellent 

b. acceptable 

c. unacceptable 

List any problems encountered • 
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8. 



9. 



Which of the followliig items In the report did you find useful? 
(Oeck as many as appropriate.) 

a. Percentiles on student performance areas 

b. Scores compared to predicted range 

c. Condition variable data 

d. Item data for condition variables 

e. Criterion-referenced Information 

f • Item data for student performance areas 

The information presented in the school report is: 

Easy to understand i ^ i . ^ ^ i Difficult to understand 

Not sufficient i i I . > f i Too extensive 

To whom have your assessment results been disseminated? (Check 
as many as appropriate.) 



a. 
b. 
c. 
d. 
e. 

f. 



school board members h. 

principals i, 

central office staff j. 

most elementary teachers k. 

most middle/ junior H.S. 1. 

teachers 
most high school teachers 
local seirvice clubs 

(Lions » Jaycees, etc.) 



PTA, PTO, any parent group 

students 

general public 

other 

none 



10. 



11. 



What methods have been used to inform others about the EQA results? (Mark 
as many as are appropriate.) 



a. faculty memorandum 

b. school district newsletter 

c . press release 

d. special written report 

e. curriculum bulletin 

f . Inservice presentations 

g. school board meeting 



h. 
1. 

J- 
k. 

1. 
m. 



faculty meeting 
PTA presentation 
special presentation 
regular meeting with 

other 
none 



Which of the following describe the use made of the EQA information? 
(Check as many as appropriate.) 

a. The Information has not, as yet, been used. 



The Informaclon was used to favorably reflect our present 

programs . 

information was used by individual principals to make 

decisions about their buildings and programs. 

4- The information served as a basis for teacher inservice 

activity. 
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e. Revisions of some existing programs are tindervay as a 

result of Inforaatlon. 
f • A new program is being planned for one or more of our 

schools as a resiilt of the information* 
A new program has been tried out" in one of our schools 

as a result of the information. 
A new program has been incorporated into one school's 

program as a result of the information. 
A new prcjram has been incorporated into several of our 

schools as a result of the information. 
j. Other: 



12. Which of the following can you attribute at least in part to 

information received from EQA? (Check as many as appropriate.) 

a. changes in course offerings 

h. changes in course content 

c. change In teaching strategies 

d. changes in teaching assignments 

changes in financial allocations 

changes in school facilities 

g. other changes: 



13. What proportion of your teachers have made use of the EQA data? 

Is greater teacher utilization of data desirable? Yes No 

If yes, how can EQA ^assist in the task? 



14. The EQA information: 

a. called attention to a problem area not previously noted 

by district staff. 

b. confirmed suspicions about district problems. 

c* did not Identify any serious problems. 

15. How do you consider the EQA program as a means of helping you make 
decisions? 

a. very useful 

b. useful 

- c. not very useful 

r-rJ^r- d. useless 
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In March, a memo offering Resource for Improvement packets was sent 
to superintendents of districts participating in the 1978 assessment. 

16. Did you or your staff request any of these packets? Yes No_ 

If yes» answer items 17 and 18 below: 

17. Were the packets used? ^ Yes N o If no, why? 



If no > do you expect to use in the current school year? 



If yes, by whom and how? 



18. Which part of the packet did you find useful? (Check as many as 
appropriate.) 1 2 ^3 U ^5 

19. What type of assistance would you like from PDE to supplement the 
initial EQA interpretation session? 



20. Comment on the value of EQA. 
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APPENDIX B 
COMMENTS ON QUESTION 3 



would you choose this type of initial interpretation in the future? 
Cominents 

- Not sure. Sometimes the visit from a representative of EQA 
has a greater Impact on the administration involved than 

another report" from someone already in the district. 

- It was very informative. 

" It gave the participants adequate background to properly 
utilize the results. t"='--^y 

- Knowledgeable persons imparted necessary facts and other 
information effectively. 

- Providing person attending workshop has knowledge of EQA and 
statistics. If not, a district meeting would be preferable. 

- Yes, for use as a general survey approach. 

' ^I^^?^^T^'^ '° question in depth any part of the assessment 
and get logical answers. 

quite a lot of questions were not well answered. 
" J^elmon? "° "in-district" 

- There Is ample time and you are away from district interruptions. 

- Too time consuming. District personnel are familiar with EOA 
format and interpretation. x«bxxx<ii. wxtn £,qa 

~ IZ^V Sroup usually generates questions and discussion which 
may be overlooked during a district interpretation. 

' attSd" ^^^^ ^^""^ opportunity for more of our people to 
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APPENDIX C 
COMMENTS ON QUESTION 5 

Would you choose this type of initial interpretation in the future? 
Why? If no, what other methods would you suggest? 

Comments 

- Convenience . 

- The personal explanation and interpretation was very valuable. 

- A very personal approach, 

- Present information to staff Cin-service?l, 

- Excellent explanation and answering of questions. 

- We received EQA results by attending the Shippensburg Conference 
This obviously was a superior way to the one-day meeting in 

the district. 

- In addition,-! would arrange for principals and teacher repre- 
sentatives to attend. 

- Good experience. We tape session. All administrators attend 
and ask questions. 

- We were able to have members of staff present, something that 
would not be possible at a regional meeting. 

- Personalized. 

- This method seems most effective to me because our administrators 
were able to ask questions and get Immediate response In terms 

of xnterpretation and significance of data. 

- Informal, relaxed atmosphere with only district personnel 
present. Zero in on more specific areas. 



APPENDIX D 
COMMENTS ON QUESTION 6 



Did you find the results provided by the new EQA Student Questionnaire 
to be: ...List any problems encountered. 



Conments 



■ Apparent problems In filling out student forms made results 
ques t lonable . 

Would have preferred answers to specific questions rather than 
groupings from items such as parental and teacher groupings. 
Would like to pinpoint the problem. 

In part, we made our own by not making the teachers complete 
their portion of the questionnaire. In part, the problem was 
the construction of that portion of the test—negatively 
worded This lack of Information altered the amount of feedback 
possxoXe. 

No problems, in fact, all questions were answered satisfactorily. 

Questions were raised in a negative fashion. Suggest that in 
the future the/ be stated In a positive manner. 

I still question some of the methods of data collection. 

In our last experience, there were unexplained "low" scores 
for our school which made Interpretation difficult. These 
unusual scores were not easily explained away, since other 
standardized testing results stayed up with other schools. 
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APPENDIX E 
COMMENTS ON QUESTION 13 



13. What proportion of your teachers have made use of the EQA data? 3 

Is greater teacher utilization of data desirable? Yes No 

If yes, how can EQA assist in the task? 

Comments 

- The packets mentioned in items #16, 17 and 18 are sufficient. 

- Local problem. 

- I'm not sure EQA can. This problem seems to be strictly a 
district circumstance. 

- Not sure at this time. 

- Should be solved within system. 

- Additional presentation to Individual buildings. 

- It is a local function. EQA should be available for consulting 
help. 

- By detailing more specifics for teachers in strategies. 

- Not sure. 

- Provide in-depth use of EQA. 

- Difficult to say. 

- By assisting local districts with in-service programs. 

- Provide additional follow-up materials. 

- By presenting an interpretation of the results to the entire 
faculty. 

- EQA need do no more. 

- Data needs to be presented in more simple terms — computer 
printouts bring out negative attitudes in many people. Also, 
data needs to allow teachers to go back and look at specific 
questions and areas without the difficulty of dealing with 
large quantities of paper. Reports could be broken into 
individual areas or subjects. 
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APPENDIX E 
CONTINUED 



Comnents 

- Provide more packets. 

- Provide assistance at In-servlce days by explaining nature and 
background of EQA to staff. 

- We nxust'do It ourselves at f>resent. 

- Not certain. 

- More teacher information and their relationships to EQA. 

- EQA cannot assist. It must be done at the local level by 
district personnel. 

- Convince staff of the validity of the testing. 

- Have member of EQA team present information to staff varied 
levels (el., ms., hs.). 

- That is a school district responsibility! 

- Principals are still working on this. 

- Elaborate on the teaching strategies booklets. 

- Explanation by people from your office. 

- Our problem is more local, i.e., to have sufficient in-service 
time and curriculum leadership to do or use tsihat is already 
there. 

- EQA should continue to develop strategies to help teachers to 
interpret the information easily and accurately. 

^ Greater breakdown of data. Permit requests for correlational 
studies. 

- District will complete assessment this year. It is a local 
thrust not necessary for additional state involvement. 

- By breaking down items according to specific curriculum 
impl icat ions . 

- Not sure* 
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APPENDIX E 
CONTINUED 



Comments 



- No definite opinion on EQA service In this regard at present. 

- Come when requested (as you do) . 

- A.V. presentations (DE produced) showing examples of teacher 
use of EQA data would be valuable. 

- Offer recommendations to improve areas of weakness that have 
been effective for other districts. 

- Representative from state meet with faculty. 

- Resource consultants. 

- Get more credibility. The data, even good results, are looked 
at skeptically. 

- Additional assistance in test interpretation. More complete 
suggestions on how to implement needed changes, e.g., citizenship, 
preparation for a changing world. 

- Information as to how to transfer EQA results into teaching 
strategies changes. 

• Workshop to share strategies and programs found to be workable 
in other schools. 

- More EQA consultants in district. More copies of results— at 
least two copies per building. 

- Local problem. 
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APPENDIX P 
COMMENTS ON QUESTION 17, PART 1 

17. Were the packets used? Y es N o If no, why? 
Comments 

- Have not had time as of this date. 

- Building principals have been sent a memo outlining what is 
available. We are awaiting requests. 

- No Interest outside the central office and the principals 
office. 

- Not received] 

- Packets requested have not been received. 

- Reserved for future in-service. 

- With a desegregation plan, the Long Range Plan, and substantial 
moving within the district, these packets were not used at the 
end of last year. 
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APPENDIX G 
COMMENTS ON QUESTION 17, PART 2 

17. If no, do you expect to use In the current school year? 
Commeots 

- Yes 

- Yes 

- Yes. Several came too late In the year to be used. They will 
be used this year, 

- If received. 

- Yes 

- No 
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APPENDIX H 
COMMENTS ON QUESTION 17 » PART 3 

If yes, by whom and how? 
Comments 

- Elementary supervisor, reading supervisor, building principals » 
coordinator for instruction. 

- Elementary supervisor and assistant superintendent. 

- Curriculxm department 

- Administrative and guidance staff 

- Principals and various building committees as well as central 
office staff 

- Counselors, administrators 

- Central offide staff. To project possible future currlculim 
change. 

- Under direction of principals and coordinators 

- Superintendent and principals 

- Teachers 

- Teachers, administrators, students 

- By principals but only superficially so far as a part of 
discussion at faculty meetings 

- Building principals in cooperation with staff members 

- Principals and teachers particularly elementary 

- Administrators, curriculxam supervisors 

- By administration and staff 

- Building principals 

- By instructional leaders, principals, and teachers to determine 
where certain modifications of curriculum might be warranted. 

- By currlcultmi coordinators and building principals in identifying 
strategies for change. By central office staff when presenting 
follow-up information to staff. 
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APPENDIX H 
CONTIMUED 



Conmients 

- Packets were distributed to principals and were used as a 
basis for discussion at dlstrlctvlde administrative meetings. 

- Utilization by principals during Interpretation meetings with 
faculty. 

- By teachers, coordinators and principals to design units, 
methods and programs to complete our curriculum revision. 

• English teachers, health and physical education teachers 

- Coordination of testing program with teachers and principals 

- Language arts - reference, social studies - reference, career 
education - proposal writing 

- Admin istrative staff. To Implement strategies for correcting 
areas of weakness determined by EQA testing. 

- Principals and superintendent for review 

- Principals working with curriculum committees 

- Packets were received by the elementary principal • Used In 
developing new Ideas In curriculum planning. 

- Cuirriculum coordinator, principals, teachers 

- Principals received appropriate copies to share with their 
staff 

Staff committees and resource people 

- Principals and staff members to determine strategies to deal 
with areas of weaknesses. 

By principals for local school In-servlce work. 
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INDEX Z 
COMMENTS ON QUESTION la 

19. What type of assistance would you like from PDE to supplement the 
Initial EQA Interpretation session? 

Conaaents 

- None at present. 

- None at the present time. 

- Any and all assistance would be appreciated. Nothing specific. 

- *As need arises, we will make contact. 
• None 

- None at this time. 

- We're well satisfied with xrfiat we've gotten. 

- Assistance is now supplemented by lU staff - adequate. 

- No specific suggestions. 

- Assistance has been available when questions arose. 

- Total inteiT>retation to the entire faculty. 

- None appears necessary « 

- No additional assistance necessary. 

- On-site visitations by PDE curriculum specialists to work with 
small groups of teachers 

- An offering of the Interpretation session at lU each time EQA 
results are available. 

- I thought the August 1978 interpretation session was "right 
on." 

- None . 

- Those areas cited in #13. 

- In areas of weakness where we would like to upgrade our program, 
the PDE could provide following: Ca) Schools that have ^jeen 
identified as having excellent programs; Cb) Available in- 
service programs and where offered; (c) Colleges offering 
seminars In the areas desired; Cd) Any other available resource 
materials. 
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INDEX I 
CONTINDED 



Coimnents 

- Continuous up*-date If possible 

- None 

- Advice as to what school districts are doing to liaprove In the 
assessment areas that are primarily affective In nature. 

- None I can think of. 

- Variable relationships; more specific Information. 

- None necessary 

- The resource packets should fill the void nicely. 

- Follow-up 

- None at this time 

- I think the EQA program functions well In Its present format. 
The Important thing has to be the availability of help ^en 
needed . 

- Provide specific suggestions and recommendations to Improve 
veak areas. 

- None 

- Presentations to school boards 

- None at the present tine 

- Continued help from Pennsylvania Department of Education would 
be appreciated, especially with the statistical interpretation 
to be given to the results. 

- None required 

- No further suggestions. Presentation was well done. 

- I think you should be congratulated on EQA and \Aat it tries 

to do. But be straight and plain speaking about its weaknesses 
so they can be considered when being used constructively. 

- On call for questions 

- PR from the PDE would be most helpful in enlisting public 
understanding of the EQA program. 
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APPENDIX I 
COiTIMUED 



Comnicnts 

- Would hope that Interpretation session would point out some 
trends and give Inferences to district personnel concerning 
results. 

- Follow-up by district visit 

• Liked Section 5 of the packets — excellent. 

- None 

Assistance requested was provided. 

- Pre-review of the test votad be helpful « 

- A follow-up visit on request would be helpful, 

- None at this time 

Workshops or training sessions for principals and subject 
supervisors to help then plan In-servlce programs. We were 
planning to attend the Executive Academy on that topic last 
year, but it did not materialize « 
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AFPENDIX J 
CGMMEKTS ON QUESTION 20 

20. Conments on the value of EQA. 

Connneiits 

- It Is difficult to compare EQA tests* l.e*, 1973-1975-1978. 

- Good experience 

- Excellent process for self assessment and to continuously 
focus on goals, long range planning and thus Improvement 

- It Is another valuable evaluation tool for the district to use 
when assessing programs. 

- The EQA has been helpful to me as a new superintendent. It 
gives me some direction toward Improvement of districts' 
problems. 

- One of the few states who provide service — very pleased « 

- Should be continually Improved^ updated and continued* 

" In and of Itself, not too bad but when tied to long range 
planning and Project 81 - trouble. 

- The validity of the results Is questionable. 

- Worthwhile • serves as a good objective assessment tool. 



- I don't have much confidence in the results on some of the 
affective goals. The Ixiformatlon in the cognitive domain and 
the condition variables has proven very helpfxil. 

- EQA allows comparative data within. expected bands - allows 
critical evaJ.xiatlon and ability to change. 

- Useful, informative. Should be continued. 

- Worthrfille 

- One of the most important additions to Pennsylvania's education 
program in years 

- We particularly appreciate two comparisons \dilch only EQA 
testing define for us; (1) The knowledge of what the average 
Pennsylvania student has achieved or feels in several areas; 
(2) The predicted range of attainment as developed from the 
many socioeconomic variables ^AdLoh are considered. 
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APPENDIX J 
CONTZMUED 



Comments 

- Extremely valuable when conducting a needs assessment 

- EQA is a valuable Instrument for loug- and short-range curriculum 
planning. However » the predicted band ranges run the danger 

of leading to an acceptance of the status quo. 

- I think there is general satisfaction with EQA. I believe it 
has a real future, especially if it can be made compatible 
with the thrust for competency requirements. I would like to 
see the latter compromised more toward the EQA movement. Many 
more local decisions can be made with EQA data than what 
appears to be the case in the «ln^^m^^| competency movement. 

- It has merit in pointing out attitudlnal areas of students and 
staff. Useful in part, has its place, but should not be over 
emphasized in evaluating the total local educational scene. 

- No additional co nunen ts 

- Worthwhile and should be continued. 

- Think It is very useful with excellent statistical treatment. 
In the hands of persons not familiar with statistics and 
correlations, some misuse of results could occur. 

- EQA was very helpful in conveying useful information about our 
schools to our parents, faculty and students. We use EQA to 
improve educational program offerings for all students. 

- Every three years the EQA is an acceptable assessment to make 
to try and correlate it with a district's achievemsnt test 
results . 

- We recommend the continuation of EQA with Improvement in data 
presentation, but definitely not toward statewide competency 
rating. ' 

~ Assists the school district in evaluating the affective areas 
in our school district and supplemental information in certain 
cognitive areas. 

- Excellent avenue for school districts to xipdate, revise or add 
to curriculum. I, as one, wish that EQA continues as a tool 
for evaltiatlon. 
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APPENDIX J 
CONTINUED 



CoosnenCs 



~ One means of data collection that, coiiibined vlth district 

data, gives a picture of district. Low percentiles disturbed 
parents more than teachers who see little value in one test 
score. 

~ We have found the EQA tests to be helpful In setting our 
priorities. The results, when Interpreted properly, are an 
aid in planning. 

- It has some value as a criterion-reference Instrument, but 
should not be the basis for a competency examination lii5)osed 
statewide, nor should it be the total basis for analysis of a 
school district. As an additional Indicator, it Is useful, 
but not a final or total Indicator. 

- I feel it's valuable for both faculty and administrators and 
should be continued. 

- Assisted us in our program development by providing data we 
many times stated but didn't have proof. EQA. provided the 
proof. 

- Provides a basis of comparison with similar districts. Provides 
an opportunity for our district to take a look at ourselves. 

- Information excellent. School districts Implementation values??? 

- Good - excellent 

- Provides an additional tool for program evaluation at the 
local level. 

~ The value is well worth the time and effort spent on it. 

- Yes, very Important, will utilize more and more. 

- Good 

- We have found it to be extremely useful and beneficial in 
pointing out areas which need "propping up," 

- Useful 

- I think EQA has been a useful tool. Our school district has 
always done well on a comparison basis—it has supported our 
premises and helped to point out deficiencies and areas of 
concern. 
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APPENDIX J 
CONTZKUED 



Connnepts 

- Value is being obscured by outsiders thru pressure groups. PR 
program is required from state and local level. Test should 
be required of everyone if it is to be used. Student options 
should not be given. 

- Elementary — indication of strengths and weaknesses with oppor- 
tunity for improvement of deficient areas. Middle School — an 
evaluative tool to assess the quality and effectiveness of a 
school program. High School — ^no value. 

- Useful as an additional monitoring devics- for a district. 

- Good pulse taker. Valuable when item analysis is pursued. 
Surprised staff with student reaction C+) to their school 
programs . 

- I believe it*s a good tool. To be effective in Long Range 
Planning, the changing of the tests should be avoided. 

- Worthwhile~We have participated three times in EQA. Although 
I understand reasons for changJLng, it would be more helpful if 
more direct comparisons could be made with previous test. 

- EQA is an attempt to set some goals for our schools. Some of 
the goals cannot be achieved by the school alone. I would 
continue EQA, and continue to simplify and improve it. 

- Excellent tool and we plan to continue utilizing EQA services. 

- None 

- Excellent program. We appreciate the information we get. 

- The EQA provides emphasis, unlike the usual standardized 
tests, in that concentration is given to probable causes of 
deficiencies, and emphasis is placed beyond achievement of 
academics, to measures related to attitudes. These factors 
are felt to substantiate the values of the EQA. 

- Of high value for comparative data. EQA should become more 
criterion-referenced as plans xjnfold for Project 81. 

- Assessment/evaluation if used in proper manner is always 
valuable tool, 

- I would like to see it continue. 



ERIC 



42 



APPENDIX J 
CONTINUED 



Conanents 

- I believe EQA to be one of the most xisefiil services which we 
receive from the Dep^irtment of Education • 

- A very good evaluation tool. It should be continued especially 
to give three-year follow through. Very helpful In reporting 
to non-^educators* 

- Tfxe EQA has served as a tool to evaluate our currlctilum offerings 
and teach strategies • Through this assessment, we have had a 
chance to view our individual schools with a perspective not 
available before. This is a third time we have participated, 
and each session has had a positive impact; i.e., we have 

added an instructional materials center to our middle school, 
partially because of thejresults of this instrument. As a 
result of this new facility, students of our middle school and 
high school have greatly benefited. We certainly appreciated 
the interpretation of the EQA^ and we implemented programs in 
the true spirit of the program. This service is certainly 
worthwhile when carried out as intended. 

- Provides an evaluation instrument to be considered when analyzing 
the curriculum; some teachers used it to reconcile behavior of 
some pupils toward their school; some teachers used it to 
satisfy their professional attitudes and concerns; some teachers 
used it to reinforce learning and/or to motivate their pupils. 

- It is helpful when trying to evalTjate parts of the school 
program that are not evaluated by our regular testing program. 
However, since the EQA changes drastically from one administra- 
tion to the next, it is difficult to get a long-range assessment 
in some areas. The other problem we have is that the students 
results fluctuate so much from one test administration to the 
next that the results are sometimes useless or at least minimized. 

- Revisions made have been most helpful, and have substantially 
improved both the utility and credibility of EQA. This is 
proving to be an increasingly useful tool in the evaluation of 
schools and their effectiveness. 

- The specifics are much more valuable than the percentiles or 
general information* 

- Excellent - please include us ir the next assessment. 
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APPENDIX J 
CONTINDED 



- EQA has flaws, but results are as valtiable as district has 
time to use them effectively. We have a long way to go to 
really use the results to their maxlsajoi. Time and administrative 
staff- the problem. 

- EQA is our only source of data for the affective areas. In 
this sense, it is serving a puirpose. 

- Excellent objectives, not enough follow^p utilizing lU potential. 

- Although I cannot point to specific changes or improvements as 
a result of EQA, I do find the entire format useful as an 
independent means of providing information from students and 
teachers. I am highly pleased with EQA as used in our latest 
participation* Data presentation was excellent* 

Excellent tool for evaluation and Improvement. 

- EQA results cause us some difficulties since our curriculum 
does not coincide with expectations of EQA testing. 

- The EQA results were valuable for our Long Range Plans and a 
means of presenting both strengths and weaknesses of our 
district. 

- We have participated in every EQA. We want to continue. We 
find them excellent and extremely helpful. 

- EQA results do not lend themselves to easy interpretation by 
the general public. 

- Another technique to take-*a-look-»see at your present offerings 
to make curricultim decisions. 

- EQA has verified findings which we had already determined and 
started appropriate changes to improve courses as well as 
content . 

- Moderate. Supplements district data* It will always be 
limited in its utility until instrument and norms allow for 
longitudinal comparisons. Matrix sampling is indefensible as 
a sampling technique with small groups of students, i.e., a 
single class of 18-21 in a grade tested* 

- EQA is of value to our district. It provides another tool to 
assist us In evaltiatlng and revising program. 
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APPENDIX J 
CONTINDED 



Comments 

- Find that EQA results, as most testing programs , confirms what 
is already known or suspected* Some difficulty encountered in 
determining effect in follow^p procedures in areas of weakness. 

- We perceive this as one of several tools in decision inaklng. 

- Excellent effort and good tool for motivation of staff* 

- In this district, we find that EQA helps us to take a more 
objective look at programs , their impacts and needs. We are 
always keenly aware of staff responses in examining attitudes 
and possible ways to Improve relationships with peers, adminis- 
trators and community. We find that our students' performance 
in the academic portions of the test correlate with scores on 
other standardized tests. We further find that it is difficult 
to institute formalized programs In citizenship per se* We 
appreciate EQA because it Is an Indicator of how our students 
respond in comparison with others across the Commonwealth in a 
variety of situations and circumstances. Results do point up 
some areas which should be improved. EQA provides a starting 
point for change for program Improvement. 

- Useful in Identifying strengths and weaknesses of our total 
educational program and making adjustments accordingly. 

- Appropriate instrument to periodically survey our community, 
programs, students and achievement. 

- Excellent for our district because the data along with standard- 
ized achievement test results have been very helpful in assessing 
the status of our recent desegregation/integration program and 
the district attendance area reorganized, 

- EQA provides data which is helpful to us in evaluating current 
curricula, planning programs and developing new curriculum. 

- Much of the information obtained through EQA would not be 
available to us through other sources. We have standardized 
test results; however, many of the goals included in EQA are 
not included in standardized tests. 
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